Improving the Mean Field Approximation via the Use of Mixture Distributions

Mean eld methods provide computationally eecient approximations to posterior probability distributions for graphical models. Simple mean eld methods make a completely factorized approximation to the posterior, which is unlikely to be accurate when the posterior is multi-modal. Indeed, if the posterior is multi-modal, only one of the modes can be captured. To improve the mean eld approximation in such cases, we employ mixture models as posterior approximations, where each mixture component is a factorized distribution. We describe eecient methods for optimizing the parameters in these models.